METHOD AND APPARATUS FOR 
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REFERENCE TO COMPUTER PROGRAM LISTING APPENDIX 

A computer program listing appendix has been submitted on two compact discs. All 
material on the compact discs is incorporated herein by reference as part of the present 
disclosure. There are two (2) compact discs, one (1) original and one (1) duplicate, and each 
compact disc includes the following ninety files: 
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ActionS et j ava 


26,409 


10/31/01 


ArmTimetOtdeirPtocessor iava 


7,095 


10/26/01 


B ayesRule j ava 


6,274 


10/26/01 


BioNET.java 


22,152 


10/24/01 


RioNetDatabase iava 


40,708 


11/1/01 






10/30/01 


BioNetTerminalException.j ava 


3,140 


8/27/01 


BioNetUtilities iava 


11,850 


10/18/01 


Classifier j ava 


47,169 


10/29/01 


ClassifierFieldManager.j ava 


8,385 


10/30/01 


ClassifierPoirnlatirvn iava 


25,894 


10/30/01 


ClassifierS et iava 


12,784 


10/30/01 


ClassifierStatistics iava 


13,778 


10/29/01 


ClassifierSysteiii.j ava 


4,248 


11/7/01 


ConditionalProbability.java 


3,433 


10/26/01 


ConditionalProbabilityMap.java 


5,566 


10/17/01 


ConditionalProbabilityMap_Doublej 


ava 


2,090 


ConditionalProbabilityMap_Integerjava 


1,760 


ConditionalProbability_Integer.j ava 


4,059 


10/26/01 


ConfigurationEvent.j ava 


3,373 


10/29/01 


ConfigurationEventListener.j ava 


899 


9/4/01 


DatabaseField j ava 


1,773 


8/27/01 


DBbioNETConfigjava 


15,479 


10/30/01 


DBcashiersjava 


3,909 


10/31/01 

x yjt ^> Ll \J L 


DBconfig.java 


4,747 


10/31/01 

x yji x / \j x 


DBdataSubsystem.java 


1,548 


11/2/01 


DBdataSubsystemF actory j ava 


9,749 


10/28/01 


DBdataSubsystemFactoryPhasel Java 


1 3,914 


10/28/01 


DBdataSubsystemFactoryPhase2.j ava 


1 3,909 


10/28/01 


DBdestmations.j ava 


4,264 


10/31/01 

X \JJ *J X / \J X 


DBintDescriptionj ava 


7,553 


10/31/01 


DBmappedNodes.java 


13,742 


10/31/01 


DBmenuItem.j ava 


41,161 


ll/6/Ol 



10/17/01 
10/17/01 
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DBmenuItemPeriod.j ava 


19,505 


11/6/01 


DBmenuItemPhasel Java 


7,273 


11/1/01 


DBmenuItemProbability.java 


7,266 


11/1/01 


DBmenuItems.j ava 


51,588 


11/6/01 


DBmenuItemsPhasel .j ava 


4,819 


11/1/01 


DBperiod.java 


14,043 


11/4/01 


DBperiodCounts.java 


5,320 


11/4/01 


DBperiods.java 


27,560 


11/6/01 


DBregisters.java 


4,228 


10/31/01 


DebugPrintNothing.j ava 


1,861 


11/2/01 


DebugPrintOut.java 


8,181 


11/2/01 


DigitalDealDatabase.j ava 


4,175 


10/31/01 


Evolvable.java 


1,301 


11/2/01 


EvolverAgentj ava 


3,676 


11/2/01 


GeneratesOffers.j ava 


1,575 


11/2/01 


HasNamedFields.j ava 


1,319 


11/2/01 


IdenticalOfferAgent.java 


3,467 


11/2/01 


IdenticalOfferlnterface.j ava 


1,376 


11/2/01 


InitializeFromResultS et.j ava 


1,336 


8/27/01 


Lcs.java 


17,294 


10/30/01 


Lcsltem.java 


2,038 


11/2/01 


LearnerAgent.java 


6,017 


11/6/01 


Learns.java 


1,637 


11/2/01 


MappedNodelnterface.j ava 


1,254 


10/18/01 


MappedNodeManager.j ava 


1,883 


10/18/01 


MapsPeriodlds.j ava 


1,202 


10/9/01 


MenuItemEvent.java 


6,027 


11/1/01 


MenuItemListener.j ava 


1,238 


11/1/01 


ObservedOutcomes.java 


1,812 


10/17/01 


Offerable.java 


2,042 


11/5/01 


Offerables.java 


3,005 


10/25/01 


OfferGeneratinglnstance.java 


4,952 


11/6/01 


OfferGenerator.java 


5,139 


10/19/01 



4 



Offerltemjava 


20,105 


11/6/01 


OfferPoolCreator.j ava 


8,324 


10/26/01 


Order.java 


15,403 


10/29/01 


Orderable.java 


1,346 


11/5/01 


Orderables.java 


1,998 


10/16/01 


Orderltemjava 


8,136 


11/5/01 


OrderProcessor.j ava 


8,737 


9/27/01 


OverDollarOfferPoolCreator.j ava 


2,173 


10/26/01 


PeriodCounts.java 


789 


11/4/01 


PeriodldMapper.j ava 


2,162 


10/26/01 


PredictionArray.java 


13,648 


10/29/01 


RefreshAgent.j ava 


1,769 


10/24/01 


RefreshListener j ava 


1,384 


11/2/01 


SqlStatement.java 


16,751 


10/24/01 


StateEvent.java 


2,986 


10/2/01 


StateEventListener.j ava 


875 


9/20/01 


SystemParameters.java 


18,660 


10/29/01 


TimerArmedOrderProcessor.j ava 


4,747 


9/26/01 


TimerThread.j ava 


1,304 


10/24/01 


Updatable.java 


1,398 


10/29/01 


Upgrade Agentj ava 


2,644 


10/25/01 


WakeUp Action.j ava 


880 


8/28/01 


Xcslnstance.j ava 


25,626 


11/6/01 


XcsOfferltemj ava 


7,860 


10/29/01 



BACKGROUND OF THE INVENTION 

Everyday, several companies spend significant sums of time and money in an effort to 
improve their operations. These efforts are manifested in various programs including 
training, communications, computer systems, product development and more. Historically, 
computerized systems have been instrumental in controlling costs and tracking performance 
within all of these disciplines. These systems have grown in flexibility and capability and, in 
general, have been perfected. Newer systems, like RetailDNA's Digital Deal™ system, are 
emerging and are now focused on driving increases in revenues and profits. Some of these 
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systems, like the Digital Deal, are rules based and often permit user modifications that can 
drive incremental performance improvements. 

Unfortunately, these systems have not had a mechanism to help change behavior or 
improve themselves over time. Therefore, the results these systems are able to produce are 
dependent upon the discipline and performance of store and senior management or systems 
support personnel. For example, if the database within a labor scheduling package is not kept 
up to date or routinely "fine tuned" it may become ineffective. 

It would be advantageous to provide a method and apparatus that overcame the 
drawbacks of the prior art. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention can change the way business practices and processes are 
improved over time. The invention may be used to improve system parameters of systems 
such as the Digital Deal™. For example, a system that provides customers with dynamically- 
priced upsell offers (defined below) may be improved to make offers that are more likely to 
be accepted. A description of systems that can provide dynamically priced upsell offers may 
be found in the following U.S. Patent Applications: 

U.S. Patent Application Serial No. 09/083,483 entitled "Method and Apparatus for 
Selling an Aging Food Product" and filed May 22, 1998; U.S. Patent Application No. 
08/920,116 entitled "Method and System for Processing Supplementary Product Sales at a 
Point-Of-Sale Terminal" and filed August 26, 1997; U.S. Patent Application Serial No. 
09/538,751 entitled "Dynamic Propagation of Promotional Information in a Network of 
Point-of-Sale Terminals" and filed March 30, 2000; U.S. Patent Application Serial No. 
09/442,754 entitled "Method and System for Processing Supplementary Product Sales at a 
Point-of-Sale Terminal" and filed November 12, 1999; U.S. Patent Application Serial No. 
09/045,386 entitled "Method and Apparatus For Controlling the Performance of a 
Supplementary Process at a Point-of-Sale Terminal" and filed March 20, 1998; U.S. Patent 
Application Serial No. 09/045,347 entitled "Method and Apparatus for Providing a 
Supplementary Product Sale at a Point-of-Sale Terminal" and filed March 20, 1998; U.S. 
Patent Application Serial No. 09/083,689 entitled "Method and System for Selling 
Supplementary Products at a Point-of Sale and filed May 21, 1998; U.S. Patent Application 
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Serial No. 09/045,518 entitled "Method and Apparatus for Processing a Supplementary 
Product Sale at a Point-of-Sale Terminal" and filed March 20, 1998; U.S. Patent Application 
Serial No. 09/076,409 entitled "Method and Apparatus for Generating a Coupon" and filed 
May 12, 1998; U.S. Patent Application Serial No. 09/045,084 entitled "Method and 
Apparatus for Controlling Offers that are Provided at a Point-of-Sale Terminal" and filed 
March 20, 1998; U.S. Patent Application Serial No. 09/098,240 entitled "System and Method 
for Applying and Tracking a Conditional Value Coupon for a Retail Establishment" and filed 
June 16, 1998; U.S. Patent Application Serial No. 09/157,837 entitled "Method and 
Apparatus for Selling an Aging Food Product as a Substitute for an Ordered Product" and 
filed September 21, 1998; U.S. Patent Application Serial No. 09/603,677 entitled "Method 
and Apparatus for selecting a Supplemental Product to offer for Sale During a Transaction" 
and filed June 26, 2000; U.S. Patent No. 6,119,100 entitled "Method and Apparatus for 
Managing the Sale of Aging Products and filed October 6, 1997. 

Further, the present invention can permit and enable other rules-based applications to 
become "self improving." 

Various embodiments of the present invention can take advantage of a multitude of 
data sources and transform these data into genetic codes or 'synthetic' DNA. The DNA is 
then used within an artificial biological environment, which the embodiments of the present 
invention can replicate. For example, each transaction may be analogized to an individual 
(species) in a population. When transactions are proven successful under certain 
environmental conditions (e.g., particular cashier or customer, time of day, day of week, 
certain store configuration, whether the destination is drive through or dine in, customer 
demographics), embodiments of the present invention can "propagate" that success. By 
culling unsuccessful transactions from the synthetic ecosystem, embodiments of the present 
invention can help eliminate undesirable transactions. Conversely, embodiments of the 
present invention can encourage the propagation of successful transactions, which drives 
incremental performance improvements. 

The following is an example of one embodiment of the present invention, offered for 
illustration only. 

RetailDNA offers a product referred to as the Digital Deal ™, which dynamically 
generates suggestive sell offers that usually include some form of value proposition (or 
discount). Customers either accept the offer or they don't. By providing results data from 
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the Digital Deal to the system described herein, overall customer accept rates and customer 
satisfaction may be improved. Each customer transaction (successful or not) can be 
translated into genetic strings or DNA. The transactions are measured as to their overall 
success ratings (success may be defined by subjectively according to any criteria) and 
includes (in this case), the percentage of customers accepting the deal and the value of the 
deal to the restaurant operator, and are propagated based upon these ratings. In this way, the 
system can exploit practices that are known to yield positive results according to various 
priorities. 

In an effort to explore new possibilities, in various embodiments the system may 
periodically create new combinations of the DNA. In the preceding example, these new 
DNA combinations are new offers that have not yet been tried or written into rules. 
Embodiments of the present invention leverage success by distributing these new ideas. The 
more information that is made available to the system, the faster the system can improve 
results. Embodiments of the present invention can spread out new ideas over many sites. In 
such embodiments, the risk and costs associated with introducing a new strand are thereby 
reduced while simultaneously gathering significant results in a short period. 

Embodiments of the present invention may also measure the actual results of both 
existing and new DNA and may continuously evolve to improve the overall effectiveness of 
the improved system. Since the whole process is automated, no human intervention is 
required to continuously improve. Thus, embodiments of the present invention can 
automatically adjust software settings to continuously generate incremental improvements in 
operational and financial performance., dramatically changing the way information systems 
affect the day-to-day operations of businesses. This may be accomplished by, e.g., creating a 
new model and method for involving and leveraging customers, systems and / or employees 
within an organization. 

The computer program listing appendix included herein describes a program which 
may be used to practice an embodiment of the present invention. 

DEFINITIONS 

The terms listed below shall be interpreted according to the following definitions in 
connection with this specification and the appended claims. 

POS terminal - a device that is used in association with a purchase transaction and 
having some computing capabilities and/or being in communication with a device having 
computing capabilities. Examples of POS terminals include but are not limited to a cash 
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register, a personal computer, a portable computer, a portable computing device such as a 

Personal Digital Assistant (PDA), a wired or wireless telephone, vending machines, 

automatic teller machine, a communication device, card authorization terminals, and / or 

credit card validation terminals. 

Offer - an offer, promotion, proposal or advertising message communicated to a 

customer at a POS terminal, including upsell offers (such as dynamically-priced upsell 

offers), suggestive sell offers, switch-and-save offers, conditional subsidy offers, coupon 

offers, rebates, and discounts. 

Upsell Offer - a proposal to a customer that he or she may purchase an additional 

product or service. For example, the customer may have an additional product or service 

added to a transaction. 

Dynamically-priced upsell offer - an upsell offer in which the price to be charged for 
the additional product depends on a round-up amount associated with the transaction. For 

example, the round-up amount may be the difference between the transaction total (the 
amount the customer is required to pay without an upsell) and the next highest dollar amount 
greater than the transaction total. According to this specific example, if the transaction total 
without the upsell is $4.25, then the round-up amount is $0.75 ($5.00-$4.25 = $0.75). In 
general, the round-up amount may also be based on the difference between any of a number 
of values associated with the transaction total and any other transaction total. For example, if 
the transaction total without the upsell is $87.50, the round-up amount may be $1 1 .50, 
resulting in a new transaction total of $99.00. Other information, such as an amount of sales 
tax associated with the transaction, may also be used to determine the round-up amount. 

Suggestive sell offer - an upsell offer in which the price to be paid for the additional 
item is a list, retail or standard price. 

Switch-and-save offer - a proposal to a customer that another product be substituted 
for (or sold in lieu of) a product already included in a transaction. In various embodiments, 
the substitute product is offered and / or sold for less than its standard price. 

Cross-subsidy offer (also referred to as a "conditional subsidy offer") - an offer to 
provide a benefit (e.g., to subsidize a purchase price, to purchase a product for a lower price) 
from a third-party merchant in exchange for the customer performing and / or agreeing to 
perform one or more tasks. For example, a customer may be offered a benefit in exchange 
for the customer (i) applying for a service offered by a third-party, (ii) subscribing to a 
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service offered by a third-party, (iii) receiving information such as an advertisement, and / or 
(iv) providing information such as answers to survey questions. 

Several embodiments of the invention will now be described with reference to the 
drawings. 

System Overview 

Fig. 1 illustrates, in the form of a block diagram, a simplified view of a POS network 
in which the present invention may be applied. 

In Fig. 1, reference numeral 20 generally refers to the POS network. The network 20 
is seen to include a plurality of POS terminals 22, of which only three are explicitly shown in 
Fig. 1. It should be understood that in various embodiments of the invention the number of 
POS terminals in the network may, for example, be as few as one, or, may number in the 
hundreds, thousands or millions. In certain embodiments, the POS terminals 22 in the POS 
network 20 may, but need not, all be constituted by identical hardware devices. In other 
embodiments dramatically different hardware devices may be employed as the POS terminals 
22. Any standard type of POS terminal hardware may be employed, provided that it is 
suitable for programming or operation in accordance with the teachings of this invention. 
The POS terminals 22 may, for example, be "intelligent" devices of the types which 
incorporate a general purpose microprocessor or microcontroller. Alternatively, some or all 
of the POS terminals 22 may be "dumb" terminals, which are controlled, partially or 
substantially, by a separate device (e.g., a computing device) which is either in the same 
location with the terminal or located remotely therefrom. 

Although not indicated in Fig. 1, the POS terminals 22 may be co-located (e.g., 
located within the same store, restaurant or other business location), or one or more of the 
POS terminals 22 may be located in a different location (e.g., located within different stores, 
restaurants or other business locations, in homes, in malls, changing mobile locations). 
Indeed, the invention may be applied in numerous store locations, each of which may have 
any number of POS terminals 22 installed therein. In one embodiment of the invention, the 
POS terminals 22 may be of the type utilized at restaurants, such as quick-service restaurants. 
According to one embodiment of the invention, POS terminals 22 in one location may 
communicate with a controller device (not shown in Fig. 1), which may in turn communicate 
with the server 24. Note that in certain embodiments of the present invention, all the 
elements shown in FIG. 1 may also be located in a single location. 
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Server 24 is connected for data communication with the POS terminals 22 via a 
communication network 26. The server 24 may comprise conventional computer hardware 
that is programmed in accordance with the invention. In various embodiments, the server 24 
may comprise an application server and / or a database server. 

The data communication network 26 may also interconnect the POS terminals 22 for 
communication with each other. The network 26 may be constituted by any appropriate 
combination of conventional data communication media, including terrestrial lines, radio 
waves, infrared, satellite data links, microwave links and the Internet. The network 26 may 
allow access to other sources of information, e.g., such as may be found on the Internet. In 
various embodiments the server 24 may be directly connected (e.g., connected without 
employing the network 26) with one or more of the POS terminals 22. Similarly, two or 
more of the POS terminals 22 may be directly connected (e.g., connected without employing 
the network 26). 

Fig. 2 is a simplified block diagram showing an exemplary embodiment for the server 
24. The server 24 may be embodied, for example, as an RS 6000 server, manufactured by 
IBM Corporation, and programmed to execute functions and operations of the present 
invention. Any other known server may be similarly employed, as may any known device 
that can be programmed to operate appropriately in accordance with the description herein. 
The server 24 may includes known hardware components such as a processor 28 which is 
connected for data communication with each of one or more data storage devices 30, one or 
more input devices 32 and one or more communication ports 34. The communication port 34 
may connect the server 24 to each of the POS terminals 22, thereby permitting the server 24 
to communicate with the POS terminals. The communications port 34 may include multiple 
communication channels for simultaneous connections. 

As seen from Fig. 2, the data storage device 30 24, which may comprise a hard disk 
drive, CD-ROM, DVD and / or semiconductor memory, stores a program 36. The program 
36 is, at least in part, provided in accordance with the invention and controls the processor 28 
to carry out functions which are described herein. The program 36 may also include other 
program elements, such as an operating system, database management system and "device 
drivers", for allowing the processor 28 to perform known functions such as interface with 
peripheral devices (e.g., input devices 32, the communication port 34) in a manner known to 
those of skill in the art. Appropriate device drivers and other necessary program elements are 
known to those skilled in the art, and need not be described in detail herein. The storage 
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device 30 may also store application programs and data that are not related to the functions 
described herein. One or more databases also may be stored in the data storage device 30, 
referred to generally as database 38. Exemplary databases that may be present within the 
data storage device 30 include a classifier database adapted to store classifiers as described 
below with reference to FIGS. 4 and 5, a genetic programs database adapted to store genetic 
programs as described below with reference to FIG. 6, an inventory database, a customer 
database and/or any other relevant database. Not all embodiments of the present invention 
require a server 24. That is, methods of the present invention may be performed by the POS 
terminals 22 themselves in a distributed and / or de-centralized manner. 

Fig. 3 illustrates in the form of a simplified block diagram a typical one of the POS 
terminals 22. The POS terminal 22 includes a processor 50 which may be a conventional 
microprocessor. The processor 50 is in communication with a data storage device 52 which 
may be constituted by one or more of semiconductor memory, a hard disk drive, or other 
conventional types of computer memory. The processor 50 and the storage device 52 may 
each be (i) located entirely within a single electronic device such as a cash register/terminal 
or other computing device; (ii) connected to each other by a remote communication medium 
such as a serial port, cable, telephone line or radio frequency transceiver or (iii) a 
combination thereof. For example, the POS terminal 22 may include one or more computers 
or processors that are connected to a remote server computer for maintaining databases. 

Also operatively connected to the processor 50 are one or more input devices 54 
which may include, for example, a key pad for transmitting input signals such as signals 
indicative of a purchase, to the processor 50. The input devices 54 may also include an 
optical bar code scanner for reading bar codes and transmitting signals indicative of the bar 
codes to the processor 50. Another type of input device 54 that may be included in the POS 
terminal 22 is a touch screen. 

The POS terminal 22 further includes one or more output devices 56. The output 
devices 56 may include, for example, a printer for generating sales receipts, coupons and the 
like under the control of processor 50. The output devices 56 may also include a character or 
full screen display for providing text and/or other messages to customers and to the operator 
of the POS terminal (e.g., a cashier). The output devices 56 are in communication with, and 
are controlled by, the processor 50. 
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Also in communication with the processor 50 is a communication port 58 through 
which the POS terminal 22 may communicate with other components of the POS network 20, 
including the server 24 and/or other POS terminals 22. 

As seen from Fig. 3, the storage device 52 stores a program 60. The program 60 is 
provided at least in part in accordance with the invention and controls the processor 50 to 
carry out functions in accordance with the teachings of the invention. The program 60 may 
also include other program elements, such as an operating system and "device drivers" for 
allowing the processor 50 to interface with peripheral devices such as the input devices 54, 
the output devices 56 and the communication port 58. Appropriate device drivers and other 
necessary program elements are known to those skilled in the art, and need not be described 
in detail herein. The storage device 52 may also store one or more application programs for 
carrying out conventional functions of POS terminal 22. Other programs and data not related 
to the functions described herein may also be stored in storage device 52. In a de-centralized 
embodiment of the invention, the storage device 52 may contain one or more of the 
previously described databases as represented generally by database 62 (e.g., a classifier 
database adapted to store classifiers as described below with reference to FIGS. 4 and 5, a 
genetic programs database adapted to store genetic programs as described below with 
reference to FIG. 6, an inventory database, a customer database and/or any other relevant 
database). 

FIG. 4 is a flowchart of a first exemplary process 400 for generating rules and/or 
offers in accordance with the present invention. As described further below, the process 400 
employs an extended classifier system ("XCS") for rule/offer generation. Extended classifier 
systems are described in Wilson, "Classifier Fitness Based on Accuracy", Evolutionary 
Computation, Vol. 3, No. 2, pp. 149-175 (1995). 

Note that while the process 400 is described primarily with reference to the generation 
of rules/offers within a quick-service restaurant ("QSR") such as McDonald's, Kentucky 
Fried Chicken, etc., it will be understood that the process 400 and the other processes 
described herein may be employed to generate rules/offers within any business setting (e.g., 
offers within a retail setting such as offers for clothing, groceries or other goods, offers for 
services, etc.). The process 400 and the other processes described herein may be embodied 
within software, hardware or a combination thereof, and each may comprise a computer 
program product. The process 400, for example, may be implemented via computer program 
code (e.g., written in C, C++, Java or any other computer language) that resides within the 
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server 24 (e.g., within the data storage device 30) and/or within one or more of the POS 
terminals 22. In the embodiment described below, the process 400 comprises computer 
program code that resides within the server 24 (e.g., a server within a QSR that controls the 
offers made by the POS terminals 22 that reside within the QSR). This embodiment is 
merely exemplary of one of many embodiments of the invention. 

With reference to FIG. 4, in step 401, the process 400 starts. In step 402, the server 
24 receives order information. For example, a customer may visit a QSR that employs the 
server 24, and place an order at one of the POS terminals 22 (e.g., an order for a hamburger 
and fries); and the POS terminal 22 may communicate the order information to the server 24. 
The order information may include, for example, the items ordered by the customer (e.g., a 
hamburger, fries, etc.) or any other information (e.g., the identity of the customer, the time of 
day, the day of the week, the month of the year, the outside temperature, the identity of the 
cashier, destination information (e.g., eat in or take out) or any other information relevant to 
offer generation). Note that order information may be received from one or more POS 
terminals and/or from any other source (e.g., via a PDA of a customer, via an e-mail from a 
customer, via a telephone call, etc.) and may be based on data stored within the server 24 
such as time of day, temperature, inventory or the like. 

In step 403, the server 24 translates the order information into a bit stream (e.g., a 
binary bit stream or sequence of bits that represent the order information). For example, each 
ordered item identifier may be translated into a predetermined number and sequence of bits, 
and the bit sequence for all ordered item identifiers then may be appended together to form 
the bit stream. Other order information such as time of day, day of week, month of year, 
cashier identity, customer identity, destination (e.g., eat in or take out), temperature, etc., 
similarly may be converted into bit sequences and appended to the bit stream. Bit streams 
may be of any length (e.g., depending on the amount of order information, the bit sequence 
lengths employed, etc.). In one embodiment, a bit stream length of 960 bits is employed. 

In one exemplary translation process, each item that may be ordered by a customer 
(e.g., each menu item), is broken down into its component parts (e.g., a hamburger equals 
beef, bread, sauce, etc.), each component part is assigned a bit sequence, and the bit sequence 
for the item is formed from a combination of the bit sequences of each component part of the 
item (e.g., beef = 1, bread = 4, sauce = 32 so that the hamburger bit sequence equals 
1+4+32=37 or 100101). Any other translation scheme may be similarly employed. To keep 
each bit stream uniform in length (e.g., to allow matching between bit streams and classifiers 
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as described below), each order is assumed to comprise a pre-determined number of items 
(e.g., six or some other number), and one or more null bit sequences may be employed within 
the bit stream if less than the number of pre-determined items are ordered. 

Once a bit stream has been generated based on the order information (step 403), in 
step 404, the bit stream is matched to "classifiers" stored by the server 24 (e.g., classifiers 
stored within the database 38 of the data storage device 30). In at least one embodiment of 
the invention, each "classifier" comprises a "condition" and an "action" that is similar to an 
"if - then" rule. That is, if the condition is met (e.g., certain items are ordered on a certain 
day, at a certain time, by a certain customer, etc.), then the action is performed (e.g., a 
customer is offered an upsell offer, a dynamically-priced upsell offer, a suggestive sell offer, 
a switch-and-save offer, a cross-subsidy offer or any other offer). In the process 400 of FIG. 
4, a bit steam is matched to a classifier by matching the bits of the bit stream with the bits of 
the classifier that represent the condition of the classifier. Methods for defining classifiers 
and for matching order information bit streams with classifiers are described in Appendix A 
herein. Note that matching may occur at the bit level, at the bit sequence level or at any other 
level. 

In step 405, the server 24 determines if a sufficient number of classifiers have been 
matched to the bit stream (determined in step 403). For example, the server 24 may require 
that at least a minimum number of classifiers (e.g., ten) match the bit stream in order to 
search as much of the available offer space as possible). Note that each matching classifier 
need not have a unique action. 

If a minimum number classifiers has not been matched to the bit stream, the process 
400 proceeds to step 406 wherein additional matching classifiers are created (e.g., enough 
additional matching classifiers so that the minimum number of matching classifiers set by the 
server 24 is met); otherwise the process 400 proceeds to step 407. Additional matching 
classifiers may be created by any technique (see, for example, process 500 in FIG. 5), and 
may be added to the "population" of classifiers stored within the server 24 (e.g., by creating a 
new database record for each additional matching classifier, or by replacing non-matching 
classifiers with the additional matching classifiers). A "reward" associated with each 
additional classifier (described below with reference to step 407) may be determined based 
on, for example, a weighted average of the reward of each classifier already present within 
the server 24. Any other method may be employed to determine a reward for additional 
matching classifiers. Following step 406, the process 400 proceeds to step 407. 
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In step 407, the server 24 determines (e.g., calculates or otherwise identifies) an 
expected reward for each matching classifier (e.g., a predicted "payoff of the action 
associated with the classifier). Rewards, predicted payoffs and other relevant factors in 
classifier selection are described further in Appendix A. 

In step 408, the server 24 determines whether it should "explore" or "exploit" the 
matching classifiers. For example, if the server 24 wishes to explore customer response (e.g., 
take rate) to the actions associated with the matching classifiers (e.g., upsell, dynamically- 
priced upsell, suggestive sell, switch-and-save, cross-subsidy or other offers), the server 24 
may select one of the actions of the matching classifiers at random (step 409). The server 24 
may choose to "explore" for other reasons (e.g., to ensure that random actions/offers are 
communicated to cashiers that may be gaming or otherwise attempting to cheat the system 
20). However, if the server 24 wishes to maximize profits, the server 24 may select the 
action of the matching classifier having the highest expected reward (step 410) given the 
current input conditions (e.g., order content, time of day, day of week, month of year, 
temperature, customer identity, cashier identity, weather, destination, etc.). 

In step 41 1, the server 24 communicates the selected action to the relevant POS 
terminal 22 (e.g., the terminal from which the server 24 received the order information), and 
the POS terminal performs the action (e.g., makes an offer to the customer via the cashier, via 
a customer display device, etc.). In step 412, the server 24 determines the results of the 
selected action (e.g., whether the cashier made the offer to the customer, whether the 
customer accepted or rejected the offer, etc.) and generates a "reward" based on the result of 
the action. Rewards are described in further detail in Appendix A. Thereafter, in step 413, 
the server 24 updates the statistics of all classifiers identified in step 404 and/or in step 406 
(see, for example, Appendix A). A classifier's statistics may be updated, for example, by 
updating the expected reward associated with the classifier. In step 414 the process ends. 

Under certain circumstances, the server 24 may wish to introduce "new" classifiers to 
the population of classifiers stored within the server 24. For example, the server 24 may wish 
to introduce new classifiers to ensure that the classifiers being employed by the server 24 are 
the "best" classifiers for the server 24 (e.g., generate the most profits, increase customer 
traffic, have the best take rates, align offers with current promotions or advertising 
campaigns, promote new products, assist/facilitate inventory management and control, reduce 
cashier and/or customer gaming, drive sales growth, increase share holder/stock value and/or 
achieve any other goals or objective). 
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FIG. 5 is a flow chart of an exemplary process 500 for generating additional 
classifiers in accordance with the present invention. The process 500 may be performed at 
any time, on a random or a periodic basis. As with the process 400 of FIG. 4, the process 500 
of FIG. 5 may be embodied as computer program code stored by the server 24 (e.g., in the 
data storage device 30) and may comprise, for example, a computer program product. 

With reference to FIG. 5, the process 500 begins in step 501. In step 502, the server 
24 selects two classifiers. The classifiers may be selected at random, may be selected 
because each has a high expected reward value, may be selected because the classifiers are 
part of a group of classifiers that match order information received by the server 24, and/or 
may be selected for any other reason. Thereafter, in step 503, a crossover operation is 
performed on the two classifiers so as to generate two "offspring" classifiers, and in step 504, 
each offspring classifier is mutated. Exemplary crossovers and mutations of classifiers are 
described further in Appendix A. An expected reward also may be generated for each 
offspring classifier (e.g., by taking a weighted average of other classifiers). In step 505, the 
offspring classifiers produced in step 504 are introduced into the classifier population of the 
server 24. For example, new database records may be generated for each offspring classifier, 
or one or more offspring classifiers may replace existing classifiers. In at least one 
embodiment, an offspring classifier is introduced in the classifier population only if the 
offspring classifier has a perceived value (e.g., an expected reward) that is higher than the 
classifier it replaces. In step 506, the process 500 ends. 

Patent applications and patents incorporated by reference herein disclose, among 
other things, a dynamically-priced upsell module (DPUM) server for providing dynamically- 
priced upsell offers (e.g., "Digital Deal" offers) to POS terminals clients. Appendix A 
illustrates one embodiment of the present invention wherein the process 400 (FIG. 4), process 
500 (FIG. 5) and/or XCS classifiers in general are implemented within a DPUM server. It 
will be understood that the present invention may be implemented in a separate server, with 
or without the DPUM server, and that Appendix A represents only one implementation of the 
present invention. 

In addition to employing XCS techniques, the present invention also employs other 
evolutionary programming techniques for generating rules and/or offers. Appendix B 
illustrates one exemplary embodiment of employing Markov and Bayesian techniques with 
genetic programs for the generation of offers within a QSR (e.g., in association with a DPUM 
server). It will be understood that the evolutionary programming techniques and other 
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methods described herein and in Appendix B may be employed to generate offers within any 
business setting (e.g., offers within a retail setting such as offers for clothing, groceries or 
other goods, offers for services, etc.). 

FIG. 6 is a flowchart of a second exemplary process 600 for generating rules and/or 
offers in accordance with the present invention. The process 600 and the other processes 
described herein may be embodied within software, hardware or a combination thereof, and 
each may comprise a computer program product. The process 600, for example, may be 
implemented via computer program code (e.g., written in C, C++, Java or any other computer 
language) that resides within the server 24 (e.g., within the data storage device 30) and/or 
within one or more of the POS terminals 22. In the embodiment described below, the process 
600 comprises computer program code that resides within the server 24 (e.g., a server within 
a QSR that controls the offers made by the POS terminals 22 that reside within the QSR). 
This embodiment is merely exemplary of many embodiments of the invention. 

With reference to FIG. 6, in step 601, the process 600 starts. In step 602, the server 
24 receives order information. For example, a customer may visit a QSR that employs the 
server 24, and place an order at one of the POS terminals 22 (e.g., an order for a hamburger 
and fries); and the POS terminal 22 may communicate the order information to the server 24. 
The order information may include, for example, the items ordered by the customer (e.g., a 
hamburger, fries, etc.) or any other information (e.g., the identity of the customer, the time of 
day, the day of the week, the month of the year, the outside temperature or any information 
relevant to offer generation). Note that order information may be received from one or more 
POS terminals and/or from any other source (e.g., via a PDA of a customer, via an e-mail 
from a customer, via a telephone call, etc.) and may be based on data stored within server 24 
such as time of day, temperature, inventory or the like. 

In step 603, the server 24 converts the order information into numerical values. For 
example, environmental information (e.g., time of day, day of week, month of year, customer 
identity, cashier identity, etc.) and order item identifiers are each assigned a numeric value 
(see Appendix B). Thereafter, in step 604, based on the order information (e.g., using the 
numerical values associated with the order information as an input), the server 24 employs 
Markov and Bayesian principles to identify associations between ordered items and other 
items that may be sold to the customer. That is, the server 24 determines all items that may 
be offered to the customer based on the customer's order (and/or all actions that may be 
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undertaken to offer items to the customer), and a "relevancy" of each item to the customer's 
order (e.g., a measure of whether the customer will accept an offer for the item). 

In step 605, the server 24 scores the potential actions (e.g., offers) that the server may 
communicate to the POS terminal that transmitted the order information to the server 24 (e.g., 
all offers that may be made to the customer). In at least one embodiment, the server 24 
scores the potential actions by assigning a numeric value to the relevancy of each item/action. 

In step 606, the server 24 determines which actions/offers may/should be undertaken 
(e.g., which offers may/should be made to the customer). For example, the server 24 may 
choose to eliminate any actions that are not profitable (e.g., upselling an apple pie for one 
penny), that are impractical or unlikely to be accepted (e.g., offering a hamburger as part of a 
breakfast meal) or that are otherwise undesirable. 

In step 607, the server 24 employs a genetic program to generate offers that are 
maximized (e.g., to pick the "best" action for the system 20). For example, the server 24 may 
generate offers/actions based on such considerations as relevancy, profit, discount 
percentage, preparation time, ongoing promotions, inventory, customer satisfaction or any 
other factors. Exemplary genetic programs and their use are described in more detail in 
Appendix B, In general, the server 24 may employ one or more genetic programs to generate 
offers/actions. In at least one embodiment, the server 24 employs numerous genetic 
programs (e.g., a hundred or more), and each genetic program is given an equal opportunity 
to generate offers/actions (e.g., based on a random selection, a "round robin" selection, etc.). 
In other embodiments, a weighted average scheme may be employed for offer/action 
generation (e.g., offers/actions may be generated based on a weighted average of one or more 
business objectives such as generating the most profits, increasing customer traffic, having 
the best take rates, aligning offers with current promotions or advertising campaigns, 
promoting new products, assisting/facilitating inventory management and control, reducing 
cashier and/or customer gaming, driving sales growth, increasing share holder/stock value, 
promoting offer deal values that are less than a dollar or more than a dollar, etc., based on 
various factors such as acceptance/take rate, average check information (e.g., to mitigate 
customer and/or cashier gaming), cashier information (e.g., how well a cashier makes certain 
offers) and/or based on any other goals, objectives or information). Filters and/or other sort 
criteria similarly may be employed. Note that weighting, filtering and/or sorting schemes 
also may be employed during the explore/exploit selection processes described previously 
with reference to FIG. 4 and process 400. 
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In step 608, the server 24 communicates the offer (or offers) to the relevant POS 
terminal 22, which in turn communicates the offer (or offers) to the customer (e.g., via a 
cashier, via a customer display device, etc.). Thereafter, in step 609, the server 24 determines 
the customer's response to the offer (e.g., assuming the cashier communicated the offer to the 
customer, whether the offer was accepted or rejected). Note that whether or not a cashier 
communicates an offer to a customer may be determined employing voice recognition 
technology as described in previously incorporated U.S. Patent Application No. 09/135,179, 
filed August 17, 1998, or by any other method. For example, it has been discovered that the 
time delay between when an offer is presented to a customer and when the offer is accepted 
by the customer may indicate that a cashier is gaming (e.g., if the time delay is too small, the 
cashier may not have presented the offer to the customer, and the cashier may have charged 
the customer full price for an upsell and kept any discount amount achievable from the offer). 

In step 610, the server 24 trains the genetic programs stored by the server 24 based on 
the results of the whether the offer was made by the cashier, accepted by the customer or 
rejected by the customer (e.g., the server 24 "distributes the reward"). Exemplary reward 
distributions are described in more detail in Appendix B. In step 611, the process 600 ends. 

As with the XCS techniques described with reference to FIG. 4 and Appendix A, new 
genetic programs may be created using crossover, replication and mutation processes. For 
example, a new population of genetic programs (e.g., offspring genetic programs) may be 
generated by "mating" (e.g., via crossover) two genetic programs, by replicating an existing 
genetic program and/or by mutating an existing genetic program or offspring genetic 
program. Selection of "parent" genetic programs may be based on, for example, the success 
(e.g., "fitness" described in Appendix B) of the parent genetic programs. Other criteria may 
also be employed. 

In at least one embodiment of the invention, a separate Markov distribution and a 
separate Bayesian distribution may be maintained for recent transactions and for cumulative 
transactions, and the server 24 may combine the recent transaction and cumulative 
transaction distributions (e.g., when making genetic program generation decisions). During 
promotions, the server 24 may choose to weight the recent transaction distributions heavier 
than the cumulative transaction distributions (e.g., to increase the response time of the system 
to promotional offers). 

The foregoing description discloses only exemplary embodiments of the invention, 
modifications of the above disclosed apparatus and method which fall within the scope of the 
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invention will be readily apparent to those of ordinary skill in the art. For instance, the 
process 400 and/or the process 600 initially may be run in the background at a store or 
restaurant to "train" the server 24. In this manner, the server 24 (via the process 400 and/or 
the process 600) may automatically learn the resource distributions and resource associations 
of the store/restaurant through observation using unsupervised learning methods. This may 
allow, for example, a system (e.g., the server 24, an upsell optimization system, etc.) to 
participate in an industrial domain, brand, or store/restaurant without prior knowledge 
representation. As transactions are observed, the performance increases correspondingly. 
This observation mode (or "self-learning" mode) may allow the system to capture 
transaction events and update the weights associated with a neural network until the system 
has been sufficiently trained. The system may then indicate that it is ready to operate and/or 
turn itself on. 

Other factors may be employed during offer/rule generation. For example, either the 
process 400 or the process 600 may be employed to decide whether an item should be sold 
now or in the fixture (e.g., based on inventory considerations, based on the probability of the 
item selling later, based on replacement costs, based on one or more other business objectives 
such as generating the most profits, increasing customer traffic, having the best take rates, 
aligning offers with current promotions or advertising campaigns, promoting new products, 
reducing cashier and/or customer gaming, driving sales growth, increasing share holder/stock 
value, promoting offer deal values that are less than a dollar or more than a dollar, etc., based 
on various factors such as acceptance/take rate, average check information (e.g., to mitigate 
customer and/or cashier gaming), cashier information (e.g., how well a cashier makes offers) 
and/or based on any other goals, objectives or information). 

Note that the genetic programming described herein may be employed to 
automatically create upsell optimization strategies evaluated by business attributes such as 
profitably and accept rate. Because this is independent of a particular retail sector, this 
knowledge can be shared universally with other implementations of the present invention 
operated in other domains (e.g., upsell optimization strategies developed in a QSR may be 
employed within other industries such as in other retail settings). Particular buying habits 
and tendencies may be 'abstracted' and used by other business segments. That is, genetic 
programs and processes from one business segment can be adapted to other business 
segments. For example, the process 400 and/or the process 600 could be used within a retail 
clothing store to aid cashiers/salespeople in making relevant recommendations to compliment 
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a given customer's initial selections. If a customer selected a shirt and pair of slacks, the 
system 20 might recommend a pair of socks, shoes, tie, sport coat, etc., depending upon the 
total purchase price of the 'base' items, time of day, day of week, customer ID, etc. 
Thereafter, the genetic programs employed by the system 20 in the retail clothing setting can 
be used across industries (e.g., genetic programs may evolve over time into a more efficient 
application). Therefore, although a given set of rules may or may not apply in another 
industry a given 'program 5 may have generic usefulness in other retail segments when 
applied to new transactional data and/or rule sets (manually or genetically generated). 

In some embodiments of the invention, unsupervised and reinforcement learning 
techniques may be combined to automatically learn associations between resources, and to 
automatically generate optimized strategies. For example, by disentangling a resource 
learning module from an upsell maximizing module, relevant, universal information may be 
shared across any retail outlet. Additionally, a reward can be specified dynamically with 
respect to time, and independently of a domain. Through the use of rewards (e.g., feedback), 
a "self-tuning" environment may be created, wherein successful transactions (offers), are 
propagated, while unsuccessful transactions are either discouraged and/or wither and die out. 
Note that rewards may also be provided to a cashier for successfully consummating an offer 
(e.g.„ if a customer accepts the reward), or for simply making offers (e.g., using voice 
technologies to track cashier compliance). The process 400 and/or the process 600 may be 
used to automatically determine (e.g., generally for all cashiers and/or specifically for 
individual cashiers) which incentive programs are most productive for motivating cashiers 
(e.g., either for a program as a whole or targeted incentives by transaction). For example, the 
present invention may be employed to determine that a cash based incentive for an entire 
team is more effective, on average, than individual incentives (or vice versa). However, it 
may also be determined that an additional individual incentive is particularly effective when 
the amount of sale exceeds a certain dollar amount (e.g. $20.00). 

In one or more embodiments, the present invention may be employed to automatically 
determine the various pricing levels within a retail outlet that has implemented a tiered 
pricing system, such as the tiered pricing system described in previously incorporated U.S. 
Patent No. 6,1 19,100. For example, the system 20 may be employed to determine the 
number (e.g., 2, 3. . . n), timing and levels of various pricing schemes. Based on consumer 
behaviors, the system 20 could become "self-tuning" using one or more of the methods 
described herein. 
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In at least one embodiment, the present invention may be employed to translate 
classifiers into "English" (or some other human-readable language). For example, humans 
(e.g., developers) may wish to understand the operation of the present invention by analyzing 
its processes and underlying assumptions (e.g., via the examination of classifiers). In this 
regard, a translation module (e.g., computer program code written in any computer language) 
may be employed that translates classifiers into a human readable form. 

Accordingly, while the present invention has been disclosed in connection with the 
exemplary embodiments thereof, it should be understood that other embodiments may fall 
within the spirit and scope of the invention as defined by the following claims. 
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APPENDIX A 



PURPOSE 

This Appendix A describes the XCS Algorithm and offers a scheme for adopting it to 
optimize the Digital Deal rules. 

OVERVIEW OF CLASSIFIER SYSTEMS 
A classifier system is a machine learning system that uses "if-then" rules, called 
classifiers, to react to and learn about its environment. Machine learning means that the 
behavior of the system improves over time, through interaction with the environment. The 
basic idea is that good behavior is positively reinforced and bad behavior is negatively 
reinforced. The population of classifiers represents the system's knowledge about the 
environment. 

A classifier system generally has three parts: the performance system, the learning 
system and the rule discovery system. The performance system is responsible for reacting to 
the environment. When an input is received" from the environment, the performance system 
searches the population of classifiers for a classifier whose "if matches the input. When a 
match is found, the "then" of the matching classifier is returned to the environment. The 
environment performs the action indicated by the "then" and returns a scalar reward to the 
classifier system. 

FIG. 7 generally illustrates one embodiment 700 of a classifier system. 

One should note that the performance system is not adaptive; it just reacts to the 
environment. It is the job of the learning system to use the reward to reevaluate the 
usefulness of the matching classifier. Each classifier is assigned a strength that is a measure 
of how useful the classifier has been in the past. The system learns by modifying the measure 
of strength for each of its classifiers. When the environment sends a positive reward then the 
strength of the matching classifier is increased and vice versa. 

This measure of strength is used for two purposes. When the system is presented with an 
input that matches more than one classifier in the population, the action of the classifier with 
the highest strength will be selected. The system has "learned" which classifiers are better. 
The other use of strength is employed by the classifier system's third part, the rule discovery 
system. If the system does not try new actions on a regular basis then it will stagnate. The 
rule discovery system uses a simple genetic algorithm with the strength of the classifiers as 
the fitness function to select two classifiers to crossover and mutate to create two new and, 
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hopefully, better classifiers. Classifiers with a higher strength have a higher probability of 
being selected for reproduction. 

OVERVIEW OF XCS 
XCS is a kind of classifier system. There are two major differences between XCS and 
traditional classifier systems: 

1 . As mentioned above, each classifier has a strength parameter that measures how 
useful the classifier has been in the past. In traditional classifier systems, this strength 
parameter is commonly referred to as the predicted payoff and is the reward that the 
classifier expects to receive if its action is executed. The predicted payoff is used to 
select classifiers to return actions to the environment and also to select classifiers for 
reproduction. In XCS, the predicted payoff is also used to select classifiers for 
returning actions but it is not used to select classifiers for reproduction. To select 
classifiers for reproduction and for deletion, XCS uses a fitness measure that is based 
on the accuracy of the classifier's predictions. The advantage to this scheme is that 
since classifiers can exist in different environmental niches that have different payoff 
levels and if we just use predicted payoff to select classifiers for reproduction then our 
population will be dominated by classifiers from the niche with the highest payoff 
giving an inaccurate mapping of the solution space. 

2. The other difference is that traditional classifier systems run the genetic algorithm on 
the entire population while XCS uses a niche genetic algorithm. During the course of 
the XCS algorithm, subsets of classifiers are created. All classifiers in the subsets 
have conditions that match a given input. The genetic algorithm is run on these 
smaller subsets. In addition, the classifiers that are selected for mutation are mutated 
in such a way so that after mutation the condition still matches the input. 
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XCS CLASSIFIERS 

A Classifier is an "if-then" rule composed of 3 parts: the "if \ the "then" and some statistics. 
The "if part of a classifier is called the condition and is represented by a ternary bitstring 
composed from the set {0, 1, #}. The "#" is called a Don't Care and can be matched to either 
a 1 or a 0. The "then" part of a classifier is called the action and is also a bitstring but it is 
composed from the set {0, 1}. There are a few more statistics (see table below) in addition to 
the Predicted Payoff and Fitness that were mentioned above. 

Example of a Classifier: 

0#011#01##000011#1 ^> 011010 

The condition (the left-side of the arrow) could translate to something like "If its Thursday or 
Tuesday at noon and the order is a Big Mac and Soda." 

The action (the right-side of the arrow) could translate to something like "Offer an ice cream 
cone." 

CLASSIFIER MATCHING 

It was stated above that the population of classifiers is searched for classifiers that match the 
input. How does a classifier match an input? First, the input from the environment (like Big 
Mac and Coke) is encoded as a string of O's and 1 's. A classifier is said to match an input if; 
1 . The condition length and input length are equal 2. For every bit in the condition, the bit is 
either a # or it is the same as the corresponding bit in the input. For example, if the input is 
"Thursday, noon, Big Mac, Soda" then there might be a classifier that has a Don't Care for 
the day of the week. If there is such a classifier then it would match the input if it also has 
"noon, Big Mac, Soda" in the condition. 

Example of Matching; 

Let the input from the environment be: 

I: 00101001 1 (Could mean something like; Thursday, 1 :00 pm, Cashier 2, Store 10, 2 Big 
Macs, 1 Large Coke) 

Let the population of classifiers be: 
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C1:01##UO##=>0110 
C2: #010#001# 1000 
C3: 0#1#100##=>0111 
C4:0#lll#0#0=>0110 
C5:0O#10O0#O=^O01O 
C6: 0##0100## => 0001 

I matches C2, C3, C6. 



CLASSIFIER STATISTICS 

The following table 1 hsts the statistics that each classifier keeps along with the algorithm for 
updating the statistics after a reward has been received from the environment. 



STATISTIC 


DESCRIPTION 


UPDATE ALGORITHM 

Let L be the Learning Rate 
Let R be the Reward received 
The "If ( experience < 1/L )" is the 
implementation of the MAM technique 


Prediction 


Keeps an average of the expected 
payoff if the classifier matches the 
input and its action is taken. Note 
that fitness is used to select 
classifiers for reproduction only. 
Prediction is used to define which 
is the "best" classifier. 


If ( experience <=1/L ) 

pred = (pred * experience + R) / 
(experience +1) 

Else 

pred = pred + L * ( R - pred ) 


Error 


Estimates the errors made in the 
prediction. 


If ( experience <= 1/L ) 

error = (error * experience + 

( | R - pred | / paymentRange)) / 
(experience + 1) 

Else 

error = error + ( L * 

((] R - pred | / paymentRange) - error)) 


Fitness 


The fitness of the classifier is based 
on the accuracy of the classifier's 
predictions. Note that fitness 
increases as error decreases. Note 


First, calculate the total accuracy for all 
classifiers in the action set. 
TotalAccuracy TA = 
□ c in Action set (numerosity c *Accuracy c ) 
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that fitness is used to select 
classifiers for reproduction only. 
Prediction is used to define which 
is the "best" classifier. 


Second, compute relative accuracy, RA. 
RA = (accuracy * numerosity) / TA. 

Then, compute fitness. 

fitness = fitness + L * (RA - fitness) 


Experience 


The number of times since its 
creation that a classifier has 
belonged to an action set. 


Increment By 1 


GA Iteration 


Denotes the time-step of the last 
occurrence of a GA in an action set 
to which this classifier belonged. 


Set to current iteration 


Action Set Size 


Estimates the average size of the 
action sets this classifier has 
belonged to. Updates to this are 
independent of updates to fitness, 
error and prediction. 


If ( experience <= 1/L ) 
size = size + 

( D c m Action Set numerosity c - size ) / 
experience 

Else 

size = size + 

L * (□ cm Acnon set numerosity c - size ) 


Numerosity 


Is the number of microclassifiers 
that are represented by this 
classifier. 


Incremented when a classifier subsumes 
another classifier and when an identical 
classifier is created. Decremented when a 
classifier is deleted from the population. If 
numerosity equals 0 then the classifier is 
deleted from the population. 


Accuracy 

TADT 1 


This is a measure of how accurate a 
classifier's predictions are. This can 
be computed from error so it does 
not need to be stored. 


Let E be the minimum error 
If ( error <=E) 
Accuracy = 1 .0 

Else 

Accuracy = 

e ({In (fallOflRate) * (error -E)/E) * faUQffRate 

Note: fallOffRate < 1 => ln(fallOffRate) < 0 
error > E => error - E > 0 
e raised to a negative power is a number 
in (0,1) so Accuracy becomes some 
number between (0,1) 



28 



INPUT COVERING - GENERATION OF MATCHING CLASSIFIERS 



When an input is received, the population of classifiers is searched and all matching 
classifiers are put in a set called the Condition Match Set, If the size of the Condition Match 
Set is less than some number N then the input is not covered. The number N is known, 
appropriately enough, as the Minimum Match Set Size and is a parameter of the system. To 
cover an input, matching classifiers are created and inserted into the population. 

The algorithm for creating matching classifiers is as follows; 

1 . Initialize the classifier, CL, so that its condition identically matches the input. 

2. For each bit in CL: Generate a random number, R 5 in [0,1]. If (R < Covering 
Probability) then change the bit to a £ #\ Covering Probability is also a parameter of 
the system, 

3 . Generate a random action that is not present in the Condition Match Set. 

4. Set the prediction equal to the mean prediction of all classifiers in the population. 

5. Set the error equal to the mean error of all classifiers in the population. 

6. Set the fitness equal to the 0.1 * mean fitness of all classifiers in the population. 

7. Set the experience equal to 0 

8. Set the GA iteration equal to the current iteration. 

9. Set the action set size equal to the mean action set size. 

10. Set the numerosity equal to 1 

11. Insert CL into the population and into the Condition Match Set 

DIGITAL DEAL CLASSIFIERS 
Digital Deal classifiers are just like regular XCS classifiers except that they have special 
requirements for matching, covering and random action generation. Both the condition and 
action contain Menu Item Ids. These are used to look up the item in the Digital Deal menu 
item database in order to get pricing and cost information. The Digital Deal classifiers are 
stored in the DPUM database. 
CONDITION 

The condition in a Digital Deal classifier is 3 64 bit chunks for the environment and 6 128-bit 
chunks for the food items. The environment contains things like day-of-week, time-of-day, 
cashier id, store id, etc. Calling the right-most bit the 0 th bit, the following table 2A defines 
the bit locations of each field in the environment: 
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Bits 


Field 


Len 


0-32 


Destination ID from DPT1M Hatnhaep 


jj 


33-44 


Month ( Jan => 1 Feb => 2 Mar=>4 etr^ nf 
Order 


1 9 


45-49 


Time of Order - Hour 


5 


64-96 


Period ID from DPTTM Hataha<ip 


D J 


97-103 


Dav Of Week fSundav => 1 Mrmrlav — > 9 
Tuesday => 4, etc) 


7 


128-159 


Register ID from DPUM database 


32 


160-191 


Cashier ID from DPUM database 


32 



* MSB is the sign bit, if set then the quantity in the remaining bits is negative 



TABLE 2A 

Each of the next 6 128-bit chunks defines a menu item. Calling the right-most bit the 0 th bit, 
the following chart defines the bit locations of each property of a menu item: 



Bits 


Property Name 


Len 


0-11 


Menu Item Type 


12 


12-23 


Size 


12 


24-35 


Temperature 


12 


36 


Pre-packaged 


1 


37 


Discounted 


1 


38-43 


Time Of Day Available 


6 


64-127 


Specific Properties for Type 


64 



The exact values for the Property Name column are defined in Appendix A-2. 
TABLE 2B 



ACTION 

An action has a variable length. The length depends on the type of action and the length of 
the binary descriptions of the menu items in the action. The shortest possible length of an 
action is 3 * 64 bits and the length will always be a multiple of 3. 
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An action is composed of groups of 3 64-bit chunks. The first chunk contains the 32-bit 
Menu Item Id from the DPUM database and the next 128-bits contain the binary description 
of that menu item. If the item is a meal then it will need more than one 128 -bit chunk for the 
description so append the additional 128-bit description with a pad of 64 O's between each 
128-bit description. 

If the action is a Replace then the first Menu Item Id is the Id of the item to replace and the 
second Menu Item Id is the Id of the offer. If the action is an Add then there will only be one 
Menu Item Id in the action. Additionally, the MSB of the first 64-bit chunk will be set if the 
action is a Replace. 

DIGITAL DEAL CLASSIFIER MATCHING 

Before an order is sent to the XCS system, it is broken up into separate meals. Exactly how 
the order is broken up is discussed later but here is an example: Let the order be 1 Big Mac, 1 
Hamburger, 2 Large Fries, 1 Coke, 1 Apple Pie then the possible meals are Ml = (Big Mac, 
Large Fries, Coke, null, null, null) and M2 = (Hamburger, Large Fries, Apple Pie, null, null, 
null), A meal contains 6 menu items. Some of the menu items may by null. A menu item 
belongs to one of 6 classes: main, side, beverage, dessert, miscellaneous, topping/condiment. 
A meal may have more than one kind of menu item in it (e.g., it is ok for a meal to have 2 
sides). The input that we are matching against is actually a meal and not an entire order. 

With all of that in mind, for a classifier, C, to match a given input, I, then all of the following 
must be true: 

1. The environments of I and C must match. The first 192 bits of C and of I are the 
environment. Use traditional bit-by-bit matching to match the two environments . 

2. Use traditional bit-by-bit matching to match the menu items. For each menu item in 
the input, there must be a matching menu item in the classifier. Order does not 
matter. The first item in the input can match, say, the third item in the classifier. 
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3. The action must match the input. For example, if the input is "Big Mac and Soda" 
then the action cannot be "Replace the small coffee with a large coffee " 

4. The amount of change must be less than the price of the offer. For example, if the 
total price of the order is $2.01 then the change is $0.99 and if the price of the offer in 
the action is $0.50 then this is not a match. This classifier could have been created for 
an order with a total price of something like $2.60 so that the action with a price of 
$.50 made more sense. 

DIGITAL DEAL RANDOM ACTION GENERATION 

The process of generating random Digital Deal actions may seem like a trivial task but is 
quite complicated. The chief culprit is the desire for the random actions to be very random. 
By "very" random, I mean that the search space of all possible actions is quite large so the 
random actions should cover as much of it as possible. The other major problem is that the 
random actions are subject to a whole slew of constraints. The actions generated should be 
profitable to both the store and the customer. For example, an offer that is not profitable to 
the store is "For your change of $0.05, add 20 Big Macs" and an offer that is not profitable to 
the customer is "For your change of $0.30, you can replace your Super-Size soda with a small 
Soda." Remember that the order is broken up into meals so random actions are generated per 
meal. 

The following is a step-by-step explanation of how random actions can be generated. 

1 . Let TP be the total price of the entire order (not just the meal). 

2. Let T be the time of day that the offer is valid (e.g., the Period ID of the order). 

3. Initialize O, the set of possible offers, to the empty set. 

4. With equal probability, randomly decide if the offer will be a replace or an add. 

5. If the offer is a replace then randomly pick something from the meal to replace. The 
item can be replaced if it's parent item is null and it's min and max price are > 0. 

6. Let TP romd be TP rounded up to the next dollar. 

7. Compute the amount of change available by subtracting TP from TP roimd . 

8. If the offer is an add then add all menu items that satisfy the following to O: the item 
is for the presently described embodiment of the invention, the min price is less than 
the change, the max price is greater than the change and the item is available in time 
period T. If the offer is a replace then add all menu items that satisfy the following to 
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O: the item is for the presently described embodiment of the invention, the price of 
the item is greater than the price of the replaced item, the (min price - min price of 
replaced) is less than the change, the (max price - max price of replaced) is greater 
than the change and the item is available in time period T. For a replace, we have to 
check both price and max price since the max price of an item may be 0 if it is not 
available as an offer. 

9. If the size of the set O generated in Step 8 is less than half the size of the minimum 
match set size (M) then add $1 to the change and return to Step 8 to try to add more 
items to O. By making the size of the offer pool greater than M, as opposed to just 
greater than 0, we are guaranteed to have more random actions. 

1 0. If the set 0 is not empty then randomly select one of the items and return it. If the set 
is empty and the offer is a replace then switch the offer to an add and go to step 8. If 
the set is empty and the offer is an add then return null; no offer will be generated for 
this order. 

XCS SYSTEM PARAMETERS 
The following TABLE 3 lists the system parameters for the XCS algorithm. An application 
with a graphical interface may be built to allow an expert user to change these parameters. 
The given defaults are the defaults recommended by the designer of the XCS algorithm (see 
Wilson 1995 referenced above). 



PARAMETER 


DESCRIPTION 


COMMON SETTING 


DEFAULT 


Population 
Size 


Number of classifiers in the 
system 


This should be large enough so 
that covering only occurs at the 
very beginning of a run. 


5000 


Action Space 
Size 


The number of possible actions 
in the system. 


It must be greater than the 
minimum match set size. 


85 


Initial 
Prediction 


The initial classifier prediction 
value used when a classifier is 
created through covering. 


Very small in proportion to the 
maximum reward. For a 
maximum reward of 1000, a 
good value for this is 10. 


10 


Initial Fitness 


The initial classifier fitness value 
used when a classifier is created 
through covering. 


0.01 


0.01 


Initial 


The initial classifier accuracy 


0.01 


0.01 
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Accuracy 


value used when a classifier is 
created through covering. 






Initial Error 


The initial classifier error value 
used when a classifier is created 
through covering. 


Should be small 


0 


Crossover 
Probability 


The probability of crossover 
within the GA 


Range of 0.5- 1.0 


0.8 


Mutation 
Probability 


The likelihood of a bit being 
mutated 


Range of 0.01 -0.05 


0.04 


Minimum 
Match Set Size 


The minimal number of 
classifiers in the match set that 
must be present or covering will 
take place 


To cause covering to provide 
classifiers for every action then 
set this equal to the number of 
available actions. 


10 


GA Threshold 


The GA is applied in a set when 
the average time since the last 
GA is greater than this threshold. 
Each classifier keeps track of a 
time stamp that indicates the last 
time that a GA was run on an 
action set that it belonged to. 
The time stamp is in units of 
"steps." 


Range 25 - 50 


25 


Covering 
Probability 


The probability of using a '#' 
symbol in a bit during covering. 


0.33 


0.33 


Learning Rate 


The learning rate for Prediction, 
Error and Fitness. Used to 
implement the MAM technique. 


0.1-0.2 


0.2 


Deletion 
Threshold 


If the experience of a classifier is 
greater than this then the fitness 
of the classifier may be 
considered in its probability of 
deletion. 


20 


20 


Exploration 
Probability 


The probability that during 
action selection the action will 
be chosen randomly. 


0.5 


0.5 


Minimum 
Error 


The error below which 
classifiers are considered to have 
equal accuracy. Used to update 
the fitness. 


0.01 


0.01 
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Fall Off Rate 


Used to update the accuracy 


0.1 


0.1 


Subsumption 
Threshold 


The experience of a classifier 
must be greater than this in order 
to be able to subsume another 
classifier. 


20 


20 


Mean Fitness 
Fraction 


Specifies the mean fitness in the 
population below which the 
fitness of a classifier may be 
considered in its probability of 
deletion. 


0.1 


0.1 


Minimum 
Reward 


The reward for a bad action. 


0 


0 


Maximum 
Reward 


The reward for a good action. 


1000 


1000 


Action Set 

Subsumption 

Flag 


Action Set Subsumption can be 
turned on/off by toggling this 
flag. 


True 


True 


GA 

Subsumption 
Flag 


GA Subsumption can be turned 
on/off by toggling this flag. 


True 


True 



TABLE 3 

SINGLE-STEP XCS ALGORITHM 



1. Let O be the order (For example, 1 KFC Meal (Chicken Leg, Cole Slaw, Beans), 1 
Chicken Sandwich, 1 Soda, and an Apple Pie). Let C be the population of classifiers. 

2. Break O into meals M b M 2 , M 3 , . . . M N 

a. Shuffle the order of the items in the order 

b. For each item in the order, find the item in the Menu Item table. If the item 
cannot be found and the item's parent is null then reject the entire order and 
return no offer. If the item cannot be found but it's parent is non-null then just 
skip the item. If the item is of type Meal (like a Extra Value Meal) then add it 
to a unique M f . If the item is not of type Meal then place it into a separate list. 
After all the items in the order have been inspected, scroll through the list of 
single type items and add those to the recently created M, or create new M it 

For the example order above the possible meals are; 

M x = Chicken Leg, Cole Slaw, Beans, Apple Pie, null, null 



M 2 = Chicken Sandwich, Soda, null, null, null 

3. For each Meal in the order, generate Condition Match Sets. Create a Condition Match 
Set by searching through the population for any classifiers that match the given Meal. 

4. If the size of any Condition Match Set is less than the Minimum Match Set Size then 
cover the Meal. See the sections on Classifiers and Digital Deal Classifiers for an 
explanation of covering. 

5. For all the Condition Match Sets, create a Prediction Array. The Prediction Array 
stores the predicted payoff for each possible action in the system. The predicted 
payoff is a fitness-weighted average of the predictions of all classifiers in the 
Condition Match Set that advocate the action. The formula for calculating the fitness- 
weighted averages is: Let AS be the set of classifiers from the Condition Match Set 

with the same action, A. Then the Predicted Payoff, P, of A is: P = (Z c e as 
Prediction, * Fitness c ) / Z c s as Fitness, 

6. If possible, choose 2 actions. The actions can be either a random selection 
(exploration) or based upon the Prediction Array (exploitation). If exploration then 
choose 2 random actions. If exploitation then choose the 2 best actions. The best 
action is defined to be the action with the highest prediction. If the highest prediction 
is shared by two or more actions then randomly choose an action. 

7. Create an Action Set for each chosen action. The Action Set is the set of classifiers 
from the Condition Match Set that have actions that match the chosen action. The 
Genetic Algorithm is run only on the Action Set. 

8. Return the actions to the environment. The amount of the reward is based on whether 
the offer was rejected or accepted. The reward is 0 if the offer was rejected. If the 
offer was accepted then the amount of the award is (1 - minPrice of offer/change in 
order) * 100 rounded to the nearest integer and then divided by 10. This gives rewards 
in the set {1000, 1 100, 1200, . . ., 2000}. This reward scheme gives accepted offers 
with bigger profits a higher reward. Since two offers are returned, the accepted offer 
is given a positive reward while the other offer is given a negative reward. 

9. Using the reward, update all the statistics of the classifiers that are part of Action Set. 
The statistics are modified in the following order: experience, action set size 
prediction, error, accuracy and fitness. Changing the order of the modifications will 
change the rate at which the system learns. For example, if prediction comes before 
error then the prediction of a classifier in its very first update immediately predicts the 
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correct payoff and consequently the prediction error is set to 0. This can lead to faster 
learning in simple processes but can be misleading in more complex problems. The 
algorithms for updating the statistics are given in a table above.Do Action Set 
Subsumption if it is enabled. In Action Set Subsumption, the Action Set is searched 
for the most general classifier that is both accurate and sufficiently experienced. All 
other classifiers in the set are tested against this general one to see if it subsumes 
them. Any classifiers that are subsumed are removed from the population. Example: 
Let the Action Set be:Cl : 01 1#1 10## — ► 0111 C2: #010#001# -> 01 1 1 C3: 
0#1#1#0## -> 01 1 1 C4: 0#1 1 1#0#0 -> 01 1 1 . C3 is the most general since it has the 
most #'s. It is more general than CI and C4. It is not more general than C2 since C2 
has a '#' in the first position and C3 does not. If C3 is accurate and sufficiently 
experienced then we could subsume CI & C4 by removing them from the population 
and increasing the numerosity of C3 by 2. 
1 1 . Run the Genetic Algorithm (GA) if the Action Set indicates that we should. The GA 
will be run on the Action Set if the average time since the last GA in the set is greater 
than the GA threshold. Average time, AT, is computed as follows: 

AT = □ □ □ DGA iteration,, * numerosity d D □ □ □ □ □ □numerosity,,) where the 

D is over the Action Set. To run the GA, use Roulette Wheel Selection to select two 

parents from the Action Set. By using Roulette Wheel selection, the classifiers with 
the highest accuracy tend to reproduce most often. Using the probability of crossover, 
the parents are crossed. If the parents are crossed then the prediction values of the 
offspring are set to the average of the prediction values of the parents. Notice that 
crossover only takes place in the condition and not in the action. Next, mutate the two 
offspring. Mutation takes place in both the action and the condition. XCS uses a 
restricted version of mutation that only allows a bit of the condition to be mutated if it 
is changed to a '#' or to a value that matches the given input. This results in an 
offspring with a condition that still matches the input. Actions are mutated as a whole 
(e.g., actions are mutated into a randomly generated new action). 
Now that we have two new offspring, check if its parent subsumes either offspring. 
The parent must have an experience level greater than the Subsumption Threshold and 
must be accurate (accuracy of 1 .0). If the offspring is subsumed then do not insert it 
into the population, just increment the numerosity of the parent. If the offspring is not 
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subsumed then it is inserted to the population. If the size of the population is greater 
than the maximum size then a classifier has to be selected for deletion. XCS uses 
Roulette Wheel Selection to select a classifier for deletion. 

ORGANIZATION OF THE SOFTWARE 
The code is organized into two parts: the Classifier System and Digital Deal Classifier. The 
Classifier System is a black box that receives a vector of bitstrings, runs the XCS algorithm 
on them, produces an action and receives rewards. It knows nothing about Digital Deal, 
QSR, Big Macs, upsells, etc. The Classifier System contains an abstract object called 
Classifier. When the Classifier System is created, it is passed the name of a classifier class. 
This classifier class encapsulates all of the peculiarities of the problem at hand. Through the 
power of inheritance, the Classifier System black box can manipulate Digital Deal classifiers 
or any other kind of classifier. The Digital Deal Classifier module supplies all the special 
routines for matching and generating random actions that were discussed above. 
CLASSIFIER SYSTEM 
SystemParameters 

Each environment must create a SystemParameters class using the function 
SystemParameters.createSystemParameters. This function verifies that the parameters are 
valid and then creates and returns a reference to a SystemParameters class. If the parameters 
are invalid then an exception is thrown. This function takes a String argument. If the 
argument is null then the default system parameters are used. If the argument is not null then 
it must be the name of a SystemParameters class. A reference to the parameters class is 
passed to the ClassifierSystem when it is created. To change the defaults: 

1 . Derive a SystemParameters class from SystemParameters. Implement the function 
localDefaultValues to add new defaults values. 

2. Pass the name of this new class to the function 
SystemParameters. createSystemParameters. 

Additional parameters can be added in a similar way. 

BitString 

A BitString is a class containing an array of longs. In Java, longs are 64-bits long. When a 
BitString is created with just a length then: 
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1 . Figure out how many 64-bit chunks are needed to contain that length. Example if 
length=65 then 2 64-bit chunks are needed. 

2. Initialize the array of longs to have a length equal to the number of chunks that was 
computed in 1. 

3. Initialize each element of the array to 0. 

When a BitString is created with a String argument then: 

1 . Do the same as above using length = string length. 

2. If the i-th character of the string is a '1' then figure out which bit in which chunk maps 
to i and set it to a 1. The mapping is from 1 -Dimension to 2-Dimensions and is given 
in TABLE 4 below. 



String Index 


Array Index 


Bit of Long 


0 


0 


0 


1 


0 


1 


63 


0 


63 


64 


1 


0 


127 


1 


63 


128 


2 


0 


i 


i/64 


i mod 64 



TABLE 4 

Each classifier is composed of two BitStrings, the condition and the action. The BitString 
class provides functions for creating BitStrings, for testing if two BitStrings are equal, for 
cloning a BitString, for accessing bits from a BitString and for modifying the bits of a 
BitString. 
ConditionBitString 

The ConditionBitString class is derived from the BitString class. This class has an additional 
array of longs which functions as a Don't Care mask. If any bit in the Don't Care mask is set 
then the corresponding bit in the original array is a Don't Care bit. The ConditionBitString 
class provides functions for determining if two ConditionBitStrings match. Using a series of 
exclusive-or operations tests matching. 
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Classifier 

A Classifier is an abstract class. In order the use the XCS package, one must derive a 
Classifier class from this parent. Implementations for the functions locallnit and clone must 
be provided. When the ClassifierSystem is created, it is given the name of the derived 
Classifier class so that any Classifiers that are created in the ClassifierSystem will be of the 
derived type. 

A Classifier has three parts: a condition, an action and some statistics. Both the condition and 
action are BitStrings. A Classifier has two constructors: the public constructor is used to 
create a Classifier with an empty condition and empty action. The function fillClassifier must 
be used to actually set the condition and action. The private constructor is only used to clone 
an existing Classifier. Functions are provided to mutate, crossover, test for equality, test for 
matching, modify the statistics, and read the statistics. 

ClassifierStatistics 

The ClassifierStatistics class encapsulates all of the classifier statistics. Functions are 
provided for accessing and modifying the statistics. The algorithms for updating the statistics 
are described in detail in the table found in the XCS Classifier Statistics section. 
ClassifierSystem 

The only interface with the outside world is through the ClassifierSystem class. One can 
create a ClassifierSystem, give an input to the system, receive an output from the system, 
give a reward to the system and query the system for the current classifier population. When 
a ClassifierSystem is created, it is given the name of the Classifier class to use when creating 
new classifiers and is given the system parameters to use in the execution of the XCS 
algorithm. 

ClassifierPopulation 

The ClassifierPopulation class contains the collection of classifiers that the XCS algorithm 
uses. Functions exist for inserting and deleting classifiers and for searching the population for 
classifiers that match an input. 
ConditionMatchSet 

The ConditionMatchSet class is used to create Condition Match Sets. A Condition Match Set 
is a collection of classifiers from the population whose condition matches a given input 
string. For traditional XCS classifiers, a classifier is said to "match" an input string if: 1. 
Condition length and input length are equal 2. For every bit in the condition, the bit is either a 
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# or it is the same as the corresponding bit in the input. Matching for Digital Deal classifiers 
is much more complicated. A Condition Match Set is said to "cover" an input if the number 
of classifiers in the match set is at least equal to some minimum number. Functions exist for 
creating the prediction array from the match set, for enumerating the match set and to test if 
the match set covers an input. 
PredictionArray 

The prediction array stores the predicted payoff for each possible action in the system. The 
predicted payoff is a fitness-weighted average of the predictions of all classifiers in the 
condition match set that advocate the action. If no classifiers in the match set advocate the 
action then the prediction is NULL. Ideally, the prediction array is an array with a spot for 
each possible action. For our system, the number of possible actions is too big so we will 
only add actions for which a classifier advocating that action exists. Functions exist for 
creating a PredictionArray from a ConditionMatchSet, for returning the best action based on 
predicted payoff and for returning a random action. The fitness-weighted average is 
computed as follows: 

1 . For a given action, compute the weighted prediction. The weighted prediction is the 
sum of the prediction * fitness for each classifier advocating that action. 

2. For a given action, compute the total fitness. The total fitness is the sum of the fitness 
for each classifier advocating that action. 

3. The fitness-weighted average for an action is the weighted prediction / total fitness. 
ActionSet 

During the course of the XCS algorithm, an action is selected from all the 
possible actions specified in the Condition Match Sets. The ActionSet class contains the set 
of classifiers from the Condition Match Set that have actions that match the selected action. 
The GA is run only on the ActionSet. For each iteration of the XCS algorithm, a new 
ActionSet is formed. If the size of the Action Set is greater than one then action set 
subsumption takes place. In action set subsumption, the Action Set is searched for the most 
general classifier that is both accurate and sufficiently experienced. If such a classifier is 
found then all the other classifiers in the set are tested against this general one to see if it 
subsumes them. Any classifiers that are subsumed are removed from the population. Setting 
the subsumption flag in the system parameters to false can disable action set subsumption. 
Since the GA is run on the Action Set, it is not obvious how this algorithm can be used with 
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historical data. Functions are included for updating all of the classifier statistics, doing action 
set subsumption, and running the genetic algorithm. 
XCS exception 

This class is the exception class for the XCS algorithm. This exception is thrown when 
functions to implement the XCS algorithm are used incorrectly. For example, an 
XCSexception is thrown if one attempts to update the prediction before updating the 
experience. 

DIGITAL DEAL CLASSIFIER 

The Digital DealClassifier class is derived from the abstract class Classifier. As stated earlier, 
Digital Deal classifiers have special requirements for generating matching classifiers, 
generating random actions and checking for matching classifiers. This class provides all of 
the special functionality. When the ClassifierSystem is created then pass the name of this 
class to it. 

INITIAL DIGITAL DEAL CLASSIFIER POPULATION 
Since XCS is capable of generating classifiers, it can start with an empty population. 
However, the learning process is much quicker if XCS is given some knowledge with which 
to start. Since Digital Deal works well, it seems logical to seed the classifier population with 
the Digital Deal rules. The Initial Rule Generator application extracts the Digital Deal rules 
from the historical order and offer data. The application can be run from the Start Menu by 
choosing DPUM>BioNET Initial Rule Generator. 

The BioNET .properties file is a flat property file that is used to configure the behavior of the 
application. The properties file can be found in c:\Program Files\DRS\DPUM\BioNET and 
can be edited with any editor. An explanation of the fields in the property file is given later. 
ALGORITHM DESIGN 

The following is a step-by-step explanation of the extraction and translation process. 

1. Create the following tables in the database: The ClassifierCondition table has 
fields: Condition, Don't Care, Action Type, Experience, Action Set Size, 
Prediction, Fitness, Numerosity, Accuracy, Error, GA Iteration, The 
ClassifierAction table has fields for the action. The ConditionAction table is 
the link table to link the condition and action. 

2. Perform the following query to extract the orders from the order table: 
SELECT OrderTable. OrderlD, Offerltem.Replace, OrderTable.DestinationID, 
OrderTable.PeriodID, OrderTable.RegisterlD, OrderTable. CashierlD, 
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OrderTable.DTStamp, OrderTable. Total, OrderltemMenuItemID, 
Orderltem.Price, Orderltern.Quantity, Offerltern.MenuIternlD, 
Offerltem. Quantity, Offerltem.OfferPrice, Orderltem.DPUMItem, 
Orderltem.ParentltemID, Offerltem.ReplaceMenuItemID FROM (Orderltem 
INNER JOIN OrderTable ON Orderltem. OrderlD = OrderTable. OrderlD) 
INNER JOIN Offerltem ON OrderTable. OrderlD = Offerltem. OrderlD 
WHERE (((0rderTable.0rderStatusID)=4) AND 
((Offerltem AcceptStatusID)=l) AND ((OrderItem.Deleted)=0)) AND 
(OrderTable.DTStamp IS NOT NULL) ORDER BY OrderTable.DTStamp 
DESC 

3. Using the first 10000 rows of the query result set, create QSRorder objects 
from all rows with the same Order ID. 

4. Translate each QSRorder into 1 or more classifiers. 

5. Add each classifier to a classifier population 

6. For each classifier in the population, add Don't Cares to the condition. 

7. For each classifier in the population, set the statistics to the default values. 

8. Write the classifier population to the database. 

MODIFYING THE RUN-TIME BEHAVIOR OF THE INITIAL RULE GENERATOR 
The InitialRules application has a property file that is used to modify its run-time behavior. 
The following TABLE 5 is an explanation of the properties in the file. 



Property Name 


Description 


Example 


jdbc. drivers 


Contains a list of class 
names for the database 
drivers. We are using the 
jdbc-odbc bridge so what 
is shown in the example 
is always valid. 


sun.jdbc.odbc.JdbcOdbcDriver 


jdbc.url 


URL of the database to 
connect to. Since we are 
using the JDBC-ODBC 
bridge, the URL will start 
with "jdbc:odbc" and the 
last part must be set with 
the ODBC Data Sources 
tool in the Control Panel. 


jdbc:odbc:McDs 
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jdbc.username 


Login ID of the user to 
log into the database 


sa 


jdbc.password 


Password needed to log 
the user into the database 




closedOrderStatusId 


Value in the 

OrderStatusID column of 
the OrderTable table that 
indicates a closed order. 


4 


accepts tatusld 


Value in the 
AcceptStatusID column 
of the Offerltem table 
that indicates an accepted 
offer. 


1 


numerosityMin 


The minimum number of 
duplicates needed for a 
rule generated from an 
order to be written to the 
database. For example, if 
set to a 1 then every order 
will be translated to a rule 
and written to the 
database. If set to a 2 
then the order must 
appear at least twice. 


4 


printClassifiers 


Set to a 1 if you want the 
rules written to standard 
output as they are written 
to the database. Set to 0 
otherwise. 


0 


printOrders 


Set to a 1 if you want the 
orders written to standard 
output as they are found. 
Set to a 0 otherwise. 


0 



TABLE 5 



Properties are entered into the property file by typing propertyName=value. There should be 
no spaces between the name, = and value. Notice that when a path and file name is given, 
the path can use forward slashes (/) or backward slashes (\) but when backward slashes are 
used they must be doubled. Java is case-sensitive so be careful. 
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TRANSLATING DIGITAL DEAL CLASSIFIERS TO ENGLISH 
Using the Translation application, Digital Deal classifiers can be translated to English. Each 
classifier is translated to a string with each field delimited with the delimiter of your choice. 
The translation can then be exported to Excel or any other spreadsheet. 

The Translator translates the Digital Deal classifiers into 3 different forms: a paragraph form, 
a parsed one-line form and into English. By far, the English version is the most useful but the 
other two forms are good for debugging. 

The paragraph form parses each field (day of week, casher id, etc) of the classifier onto a 
separate line. The following is an example of one classifier translated into paragraph form: 

CONDITION 

ENVIRONMENT 

Day of Week: 10#0#00 

Period ID: 000#####000#00000##00####000000#0 

Month: 00000000 100# 

Time of Day - Hour: ##001 

Cashier ID: 00#000000##0##000000000##0#####0 

Register ID: 000#00000000000#00000##0#00001## 

Destination ID: 0000###0#00#0#0###0##000000#0#0## 

ITEM 1 

Type: 0000#00###00 

Size: 000000000010 

Time of Day Available: #001 10 

Discounted: 0 

Prepackaged: 0 

Temperature: ####000##001 

Side: 0000##00##00000#0##0##0#0#0000000001##00000#00###00###00#00#0000 
ITEM 2 

Type: 0000##0000## 

Size: 0###000##000 

Time of Day Available: 00#000 

Discounted: 0 

Prepackaged: # 

Temperature: 0#000##00000 

Empty-Item: ##00#0#000#0000#000###0#0#00#0000#0000##0000000#0##000#000#0#000 

ITEM 3 

Type: 000000#00##0 
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Size: 000000###0#0 
Time of Day Available: 000000 
Discounted: # 
Prepackaged: 0 
Temperature: ##000#0000## 

Empty-Item: 00000#0000000000000000#000000000###0000000###0##0#000#00#000#### 

ITEM 4 

Type: 00#00##0###0 

Size: 0000000000## 

Time of Day Available: #0##00 

Discounted: 0 

Prepackaged: 0 

Temperature: 000#0####00# 

Empty-Item: 0000000000#0#0#000##000000#000##000##00##0000#000000#00##0###00# 

ITEM 5 

Type: 0##00##0##0# 

Size: 00000#000#0# 

Time of Day Available: 00#00# 

Discounted: 0 

Prepackaged: 0 

Temperature: 0#0000000### 

Empty-Item: 000#0#00#00000000##0#0000#00##00#0###000#000000##00#00#0#0#00#00 

ITEM 6 

Type: 0#0#000000## 

Size: #0##0000#0## 

Time of Day Available: 0#0000 

Discounted: 0 

Prepackaged: 0 

Temperature: 000#00000000 

Empty-Item: #0000#0#000000000#0#00#####0#000#00#0000000#000#00#00#0##0000#00 

ACTION 

Action-Type: REPLACE 

REPLACED ITEM 

ITEM 1— 

Menu Item Id: 1 1 

Type: 000000000100 

Size: 000000000010 

Time of Day Available: 000110 

Discounted: 0 

Prepackaged: 0 
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